Method for realizing user interface using camera and mobile communication terminal for the same

ABSTRACT

A method for realizing a user interface using a camera module and a mobile communication terminal for the same. If a user makes a predetermined motion in a state in which the camera module of the mobile communication terminal is activated, the mobile communication terminal performs a predetermined action according to the motion pattern by recognizing the user motion and patterning the motion. In this case, the action performed according to the motion pattern corresponds to mouse control in a mouse mode, game control in a game mode, and character input in a character input mode.

PRIORITY

This application is a Continuation Application of U.S. application Ser.No. 14/221,966, filed in the U.S. Patent and Trademark Office on Mar.21, 2014, which is a Continuation Application of U.S. application Ser.No. 11/471,359, filed in the U.S. Patent and Trademark Office on Jun.20, 2006, and issued as U.S. Pat. No. 8,694,053, and claims priority toan application entitled “Method for Realizing User Interface UsingCamera and Mobile Communication Terminal For The Same” filed in theKorean Intellectual Property Office on Jun. 20, 2005 and assigned SerialNo. 10-2005-0053254, the content of each of which are incorporatedherein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a method for realizing a user interfaceusing a camera module and a mobile communication terminal for the same.

2. Description of the Related Art

Recently, as a mobile communication terminals have become wide spread, avariety of services have become available such as a traffic informationservice for informing a user who is driving a vehicle of trafficconditions through the screen of the mobile communication terminal, aMYBELL service (enabling the user to register the user's own ringingsound in the mobile communication terminal and to use the ringingsound), and a mobile telephone call service for notifying the arrival ofa message on a voice mail-box. The user of the mobile communicationterminal can receive a variety of convenient functions in addition to anintrinsic telephone communication function because the mobilecommunication terminal is designed to receive a variety of additionalservices, such as a voice mail-box function and a specificinformation-receiving function, from a service provider.

As such services and additional functions have become increasinglyexpanded, a mobile communication system provides an Integrated ServicesDigital Network (IDSN)—level multimedia service with a data rate ofseveral Mb/s, as well as voice and data. A terminal adaptable for themobile communication system described above has been developed. A mobilecommunication terminal equipped with a camera capable of storing animage obtained by photographing a subject is introduced as one exampleof the development.

As the mobile communication terminal becomes popularized, a user cantake a picture regardless of time and location using the mobilecommunication terminal. In addition, since the mobile communicationterminal always carried by a user is equipped with a camera, the usermay photograph an emergent situation or may take a picture whenever itis required.

Accordingly, it is desirable to utilize the mobile communicationterminal equipped with the camera with more effect and through variousadditional schemes based on the above described advantages. In addition,it is necessary to add a variety of services and additional functions tothe mobile communication terminal equipped with the camera in order toenhance the usage efficiency of the mobile communication terminal andsatisfy a variety of user demands. For example, an optical characterrecognition (OCR) function enables the mobile communication terminal torecognize phone numbers and data printed in a document by using thecamera of the mobile communication terminal.

However, although a variety of applications are available when themobile communication terminal is equipped with the camera, the use ofthe camera of the mobile communication terminal is limited for somefunctions, such as the OCR function.

SUMMARY OF THE INVENTION

Accordingly, the present invention has been made to solve theabove-mentioned problems occurring in the prior art, and an object ofthe present invention is to provide a method for realizing a userinterface using a camera module and a mobile communication terminal forthe same.

To accomplish the above objects, an apparatus is provided that includesa processor-implemented controller that identifies an applicationactivated in relation with the apparatus, identifies a gesture using acamera operatively coupled with the apparatus, performs a first functionin response to the gesture if the application is a first application,and performs a second function in response to the gesture if theapplication is a second application.

Also provided is a method that identifies, at an electronic device, anapplication activated in relation with the electronic device, identifiesa gesture using a camera operatively coupled with the electronic device,performs a first function in response to the gesture if the applicationis a first application, and performs a second function in response tothe gesture if the application is a second application.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects, features and advantages of the presentinvention will be more apparent from the following detailed descriptiontaken in conjunction with the accompanying drawings, in which:

FIG. 1 is a block diagram illustrating a structure of a mobilecommunication terminal according to an embodiment of the presentinvention;

FIG. 2 is a flowchart illustrating a method for realizing a userinterface using a camera module according to a first embodiment of thepresent invention;

FIG. 3 is a flowchart illustrating a method for realizing a userinterface using a camera module according to a second embodiment of thepresent invention; and

FIG. 4 is a flowchart illustrating a method for realizing a userinterface using a camera module according to a third embodiment of thepresent invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

Hereinafter, preferred embodiments of the present invention will bedescribed in detail with reference to the accompanying drawings. Notethat the same or similar components in drawings are designated by thesame reference numerals as far as possible although they are shown indifferent drawings. In the following description of the presentinvention, a detailed description of known functions and configurationsincorporated herein will be omitted when it may make the subject matterof the present invention unclear.

According to an embodiment of the present invention, it is possible torealize a user interface module using a camera module in a mobilecommunication terminal. If a user makes a predetermined motion during astate in which the camera module of the mobile communication terminal isactivated, the mobile communication terminal recognizes and patterns theuser motion and performs a predetermined action according to the motionpattern.

The action performed according to the motion pattern may correspond tothe control of a mouse in a mouse mode or the control of a game in agame mode. In addition, the action may correspond to the input of acharacter in a character inputting mode.

Hereinafter, a block diagram illustrating the structure of a mobilecommunication terminal will be described with reference to FIG. 1.

FIG. 1 is a block diagram illustrating a structure of a mobilecommunication terminal 100 according to an embodiment of the presentinvention. The mobile communication terminal 100 includes a RadioFrequency (RF) module 110, a base band processing module 120, a userinterface module 130, a audio processing module 140, a controller 150, amemory 160, a display 170, a motion recognition module 180, a cameramodule 190, and a character recognition module 200.

The controller 150 of the mobile communication terminal 100 providesoverall control for the mobile communication terminal. In addition, thecontroller 150 controls each module shown in FIG. 1 in order to realizea user interface for operating the camera module according to apreferred embodiment of the present invention.

In a more detailed description of the operation of the controller 150,the controller 150 activates the camera module 190, recognizes a usermotion through the camera module 190, and patterns the user motion.

The controller 150 reads out a mouse controlling command correspondingto a user motion pattern from the memory 160 in the mouse mode accordingto a first embodiment of the present invention. The user moves his/herhand according to a motion pattern within the photographic field of thecamera module 190 of the mobile communication terminal 100. The motionpattern is preliminarily stored in the memory 160 and a control commandcorresponding to the motion pattern is also stored in the memory 160. Ifthe user carries out the action of a predetermined motion pattern basedon the storage of the motion pattern, the mobile communication terminalperforms the control command corresponding to the motion pattern. Forexample, the user may carry out a click action or roll a mouse wheel asif the user actually handles the mouse. The controller 150 performs acontrol action such that the mouse control command, identical to thatwhich occurs when the user actually clicks the mouse or rolls the mousewheel, is issued with respect to the corresponding motion.

The controller 150 reads out from the memory 160 a game controllingcommand corresponding to a user motion in the game mode according to asecond embodiment of the present invention. For example, if the usermakes motions such as gripping, hitting, or moving by using his/her handas if the user were actually playing a game, the controller 150 patternsthe user motion and reads out from the memory 160 a game controllingcommand corresponding to the user motion pattern. The game controllingcommand is stored in the memory 160 in the same way that the motionpattern, corresponding to the game controlling command, is stored in thememory 160.

In addition, the controller 150 according to a third embodiment of thepresent invention reads out a character according to a tracecorresponding to the user motion pattern so as to display the characteron the display 170 in the character input mode. The user moves his/herhand within the photographic field of the camera module 190 of themobile communication terminal 100 such that the motion pattern iscreated. For example, the user actually may move his/her hand as ifhe/she actually writes characters.

Then, the controller 150 reads out from the memory 160 a characteraccording to the trace corresponding to the user motion pattern so as todisplay the character on the display 170. This character inputtingfunction is mainly used for a Short Message Service (SMS) messagetransmitting function, a memo pad, etc.

The camera module 190 photographs user movement (i.e., user motion)within the photographic field so as to provide the photographed image ofthe user motion to the controller 150, the motion recognition module180, and the character recognition module 200. The motion recognitionmodule 180 recognizes and patterns the user motion and then provides theuser motion to the controller 150. In addition, the characterrecognition module 200 creates a trace corresponding to the user motionso as to provide the user motion to the controller 150.

The memory 160 includes a read only memory (ROM), a random accessmemory, and a voice memory used for storing a plurality of programs andinformation required when the operation of the mobile communicationterminal 150 is controlled. In addition, the memory 160 stores controlcommands according to at least one motion pattern and charactersaccording to predetermined traces according to the present invention.

In addition, the RF module 110 transmits/receives an RF signal to/from abase station through an antenna. The RF module 110 converts a receivedsignal into an intermediate frequency (IF) signal and outputs theconverted signal to the base band processing module 120. The RF module110 converts an IF signal input from the base band processing module 120into an RF signal and transmits the RF signal. In addition, the baseband processing module 120 is a base band analog Application SpecificIntegrated Circuit (ASIC) providing an interface between the controller150 and the RF module 110. The base band processing module 120 convertsa base band digital signal input from the controller 150 into an analogIF signal so as to apply the analog IF signal to the RF module 110, andconverts an analog IF signal input from the RF module 100 into a baseband digital signal so as to apply the base band digital signal to thecontroller 150.

The audio processing module 140 outputs received audio data afterconverting the audio data into audible sound through a speaker under thecontrol of the controller 150. The audio processing module 140 convertsan audio signal input from a user through a microphone into anelectrical signal to be transmitted to the controller 150. The userinterface 130 includes a plurality of numeric keys and functions andoutputs key-input data corresponding to a key pressed by the user to thecontroller 150. The display 170 generally includes a liquid crystaldisplay (LCD) and displays a variety of messages and icons under thecontrol of the controller.

FIGS. 2 to 4 are flowcharts illustrating a control procedure employedfor realizing the user interface using the camera module according tothe embodiments of the present invention, and FIG. 2 illustrates a casein which the user interface is realized using the camera module in amouse mode.

Referring to FIG. 2, under the mouse mode, the controller 150 displays acursor on a screen of the display 170 and activates the camera module190 in step 310. In this case, the cursor may be displayed or not. Then,the controller 150 determines if a user motion is input in step 320. Ifthe user motion is input, the controller 150 goes into step 330 andcontrols the motion recognition module 180 to pattern the user motion.If the motion recognition module 180 patterns the user motion to beprovided to the controller 150, the controller 150 in step 340 reads outfrom the memory 160 a mouse controlling command corresponding to themotion pattern. Thereafter, the controller 150 executes the mousecontrolling command according to the read-out mouse controlling commandin step 350. For example, if the user makes a mouse click motion usinghis/her hand as if the user actually clicks the mouse, the controller150 performs a control motion corresponding to a case in which the userclicks a menu item or an Internet icon for which a cursor displayed onthe screen of the display 170 is positioned. If the user performs amotion of rolling the mouse wheel as if the user actually rolls themouse wheel, the controller 150 moves the cursor displayed on the screenof the display 170 correspondingly to the mouse wheel turning motion.

FIG. 3 illustrates a case in which the user interface is realized usingthe camera module in a game mode.

Referring to FIG. 3, under the game mode, the controller 150 activatesthe camera module 190 in step 410. Thereafter, the controller 150determines if the user motion is input in step 420. If the user motionis input, the controller 150 goes into step 430 and controls the motionrecognition module 180 to pattern the user motion. If the motionrecognition module 180 patterns the user motion to be provided to thecontroller 150, the controller 150 in step 440 reads out from the memory160 a game controlling command corresponding to the motion pattern.Thereafter, the controller 150 executes the game controlling commandaccording to the read-out game controlling command in step 450. Forexample, if the user makes motions such as gripping, hitting, or movingby using his/her hand as if the user is actually playing a game, thecontroller 150 reads out from the memory 160 the game controllingcommand for a corresponding motion and executes the game controllingcommand.

FIG. 4 illustrates a case in which the user interface is realized usingthe camera module in a character inputting mode.

Referring to FIG. 4, under the game mode, the controller 150 activatesthe camera module 190 in step 510. Thereafter, the controller 150determines if a user motion is input in step 520. If the user motion isinput, the controller 150 goes into step 530 and controls the characterrecognition module 200 to create the trace corresponding to the usermotion. If the character recognition module 200 creates the tracecorresponding to the user motion to be provided to the controller 150,the controller 150 reads out from the memory the appropriate one of 160available characters corresponding to the trace in step 540 and displaysthe read-out characters in step 550. This character inputting functioncan be usefully used when a Short Message Service (SMS) message iswritten or when a schedule is made.

As described above, according to the present invention, a user interfacemodule is realized by using a camera module included in a mobilecommunication terminal, so that it is possible to expand an applicationfunction of the camera module and improve user convenience for the usageof the mobile communication terminal.

While the invention has been shown and described with reference tocertain preferred embodiments thereof, it will be understood by thoseskilled in the art that various changes in form and details may be madetherein without departing from the spirit and scope of the invention.Consequently, the scope of the invention should not be limited to theembodiments, but should be defined by the appended claims andequivalents thereof.

What is claimed is:
 1. An apparatus comprising: a processor-implementedcontroller configured to: identify an application activated in relationwith the apparatus; identify a user motion from at least one imagecaptured through a camera operatively coupled with the apparatus;determine a gesture corresponding to the user motion; perform a firstfunction in response to the gesture if the application is a firstapplication; and perform a second function in response to the gesture ifthe application is a second application, wherein the identifying of theapplication comprises activating the camera based at least in part on adetermination that the application is a first type of application, andrefraining from activating the camera based at least in part on adetermination that the application is a second type of application, andwherein the processor-implemented controller is further configured topresent, via a display operatively coupled with the apparatus, anindication associated with to the gesture.
 2. The apparatus of claim 1,wherein a corresponding one of the first function and the secondfunction is to be performed as if the gesture were an input received viaa mouse associated with the apparatus.
 3. The apparatus of claim 1,wherein a corresponding one of the first function and the secondfunction is to be performed as if the gesture were an input received viaa key associated with the apparatus.
 4. The apparatus of claim 1,wherein the processor-implemented controller is configured to: move, viathe display, the indication from a first location to a second locationin response to the gesture.
 5. The apparatus of claim 1, wherein theprocessor-implemented controller is configured to: select a menu or anicon in relation with the application as at least part of acorresponding one of the first function and the second function.
 6. Theapparatus of claim 1, wherein the processor-implemented controller isconfigured to: determine a character corresponding to the gesture as atleast part of a corresponding one of the first function and the secondfunction.
 7. The apparatus of claim 6, wherein the controller isconfigured to: provide the character to the application as an input. 8.The apparatus of claim 6, wherein the controller is configured to:transmit the character to an electronic device external to theapparatus.
 9. The apparatus of claim 1, wherein at least one of thefirst application or the second application comprises a gameapplication, a message application, a memo application, an internetapplication, or a schedule application.
 10. A method comprising:identifying, at an electronic device, an application activated inrelation with the electronic device; identifying a user motion from atleast one image captured through a camera operatively coupled with theapparatus; determining a gesture corresponding to the user motion;performing a first function in response to the gesture if theapplication is a first application; and performing a second function inresponse to the gesture if the application is a second application,wherein the identifying of the application comprises activating thecamera based at least in part on a determination that the application isa first type of application, and refraining from activating the camerabased at least in part on a determination that the application is asecond type of application, and wherein a corresponding one of the firstfunction and the second function being performed comprises presenting,via a display operatively coupled with the electronic device, anindication associated with the gesture.
 11. The method of claim 10,wherein a corresponding one of the first function and the secondfunction is performed as if the gesture were an input received via amouse associated with the electronic device.
 12. The method of claim 10,wherein a corresponding one of the first function and the secondfunction being performed comprises: moving, via the display, theindication from a first location to a second location in response to thegesture.
 13. The method of claim 10, wherein a corresponding one of thefirst function and the second function being performed comprises:determining a character corresponding to the gesture.
 14. The method ofclaim 13, wherein the corresponding one of the first function and thesecond function being performed comprises: providing the character tothe application as an input.
 15. The method of claim 10, wherein acorresponding one of the first function and the second function beingperformed comprises: providing, via an output device operatively coupledwith the electronic device, a user interface in relation with theapplication, the user interface representing an action corresponding tothe gesture.
 16. A non-transitory machine-readable storage devicestoring instructions that, when executed by one or more processors,cause the one or more processors to perform operations comprising:identifying an application activated in relation with an electronicdevice; identifying a gesture using a gesture sensor operatively coupledwith the electronic device; performing a first function in response tothe gesture if the application is a first application; and performing asecond function in response to the gesture if the application is asecond application, wherein the identifying of the application comprisesactivating a camera based at least in part on a determination that theapplication is a first type of application, and refraining fromactivating the camera based at least in part on a determination that theapplication is a second type of application, and wherein a correspondingone of the first function and the second function being performedcomprises presenting, via a display operatively coupled with theelectronic device, an indication associated with the gesture.
 17. Anapparatus comprising: a camera; a display; and a processor-implementedcontroller operatively coupled to the camera and the display, configuredto: identify an application activated in relation with the apparatus,identify a user hand motion within a photographic field of the camera,control display on the display of an indication corresponding to themotion of the user hand, perform a first function in response to themotion of the user hand if the application is a first application, andperform a second function in response to the motion of the user hand ifthe application is a second application, wherein the identifying of theapplication comprises activating the camera based at least in part on adetermination that the application is a first type of application, andrefraining from activating the camera based at least in part on adetermination that the application is a second type of application, andwherein the processor-implemented controller is further configured topresent, via the display, an indication associated with the gesture.